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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

I) K1 Responsive to communication(s) filed on 19 October 2007 . 
2a)Q This action is FINAL. 2b)E3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) KI Claim(s) 1-18 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-18 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)Q accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

I I) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

!.□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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The amendment filed on October 19, 2007 is entered and made of record. 

Response to Applicant's Arguments 

1 . In the interview on October 4, 2007, the examiner agreed that presented prior 
arts, Tomasi (2004/0005092) and Ishiyama (2002/0097906), do not disclose a unit 
determining a straight line which passes through a measuring point on the object and a 
specific point on said camera from the image of the object captured by said camera. 
However, in an updated searched shows Gilliland (US pat no 6,101,268) discloses a 
unit determining a straight line which passes through a measuring point on the object 
and a specific point on said camera from the image of the object captured by said 
camera (see figure 4B, R1-1 and R1-2 is read as the straight line). 

2. In the interview on October 4, 2007, the examiner agreed that presented prior 
arts, Tomasi (2004/0005092) and Ishiyama (2002/0097906), do not disclose 
determining parameter values that describe a transformation expressing geometrical 
deformation with respect to said reference image provided by mapping using camera. 
Okajima (US pat no 5,917,940) discloses determining parameter values that describe a 
transformation expressing geometrical deformation with respect to said reference image 
provided by mapping using camera (see figure 6, element 34 calculates the deformation 
of the image input, the image input unit map the image that is captured). 
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Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claim 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Higuchi 
(US pat no 5,129,010) in view of Ishiyama (US pub no 2002/0097906) and Gilliland 

('268). 

With regards to claim 1, Higuchi discloses a three-dimensional sensor comprising: 

pattern light projecting unit for projecting a slit light or a spot light onto a surface 

of an object (see figure 1 , element 1 is a slit light unit projecting slit light onto an object); 
a camera that captures a two-dimensional image of the object (see figure 1 , 

element 3); 

a unit causing said camera to capture the slit light or the spot light projected by 
said pattern light projecting unit onto the surface of the object, and determining the 
surface of the object on which said measuring point exists (see figure 1, element 7 is 
the slit light detecting unit; see column 9, lines 4 to 12, discusses the center position of 
the slit is obtained, the center point of the slit light is read as the measuring point); and 

a unit determining a three-dimensional position of said measuring point and the 
surface determined by said unit for determining the surface of the object (see column 9, 
lines 4 to 12; figure 17, the shape of the surface is determined by the unit in figure 16). 
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Higuchi does not disclose a unit determining a straight line which passes through a 
measuring point on the object and a specific point on said camera from the image of the 
object. Gilliland discloses a unit determining a straight line which passes through a 
measuring point on the object and a specific point on said camera from the image of the 
object captured by said camera (see figure 4B, R1-1 and R1-2 is read as the straight 
line). One skilled in the art would include determining a straight line which passes 
through a measuring point on the object and a specific point on said camera from the 
image of the object because to use triangulation to find the location of the cameras in 
three dimensional space and determine the shape of the object, to improve recognition 
of the object. 

Higuchi and Gilliland do not disclose a unit calculating an amount of rotation of the 
object around said measuring point on a plane including said surface of the object by 
comparing a shape of the image of the entire object or part of the object captured by 
said camera with a prepared reference shape of the entire object of part of the object. 
However, Ishiyama discloses a unit calculating an amount of rotation of the object 
around said measuring point on a plane including said surface of the object by 
comparing a shape of the image of the entire object or part of the object captured by 
said camera with a prepared reference shape of the entire object of part of the object ( 
see paragraph 184, where R x , R y and R z are rotation information of a target object, see 
paragraph 187 is where it the rotational information are obtained, the feature points are 
read as the measuring point, where the feature points are use to identifying the face 
image; in the specification of the current invention on page 13, lines 21 to 25 defines the 
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rotation information are by the sides of an object, in Ishiyama, if any of the rotational 
information not zero than the face image must be rotated by some degree depending on 
the value the rotational parameters). One skilled in the art would include calculating 
rotational information because information by the side of the object might show more 
details and features leading to a more accurate recognition system. 

3. Claims 2 - 4 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ozeki (US pat no 5,280,542) in view of Okajima (US pat no 5,917,940) and official 
notice (MPEP 2144.03). 

With regards to claim 2, Ozeki discloses a three-dimensional visual sensor which 
performs a three-dimensional measurement of an object, comprising: 

a two-dimensional information acquiring unit, wherein said two dimensional 
information acquiring unit determines a position of a measuring point of said object on a 
two-dimensional image including said object captured by a camera (see figure 1 , 
element 14 is the two dimensional acquiring unit, see column 3, lines 37 to 43 where it 
discusses determining the position of the slit light in the image); 

a three-dimensional information acquiring unit receives a reflected light of a light 
projected by projecting unit onto said object by unit of light receiving unit to acquire 
three-dimensional information on inclination of surface on which said measuring point of 
said object exists and/or a distance from said camera to the surface (see figure 1 , 

j 
j 
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element 22 detects slit light, see figure 8 where it shows a graph of the intensity of the 
slit light, where the intensities varies showing up and down inclination); and 

an information combining unit, wherein said information combining unit combines 
information acquired by said two-dimensional information acquiring unit and information 
acquired by said three dimensional unit of said camera and generates new three- 
dimensional information (see figure 1 , the slit line detecting area, detecting the surface 
shape of the object is read as the three-dimensional information and the image taken by 
the 'TV camera' is read as the two dimensional image, 38, the vertical imaging position 
detecting circuit detects the two-dimensional image in the vertical direction with its 
output going to the lookup table, the lookup table is the final determined position of 
object of the image, the information from the slit line detecting unit are processed 
through elements 30, 32 and 38' then is inputted to the lookup table, Ks is read as the 
three dimensional information combined with Ls, where it is read as the two-dimensional 
information). 

Ozeki does not disclose comparing a reference image including a characteristic area of 
the object with an image of said characteristic area in said two-dimensional image and 
determines parameter values that describe a transformation expressing geometrical 
deformation with respect to said reference image provided by mapping using said 
camera. Okajima discloses comparing a reference image including a characteristic area 
of the object with an image of said characteristic area in said two-dimensional image 
and determines parameter values that describe a transformation expressing geometrical 
deformation with respect to said reference image provided by mapping using said 
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camera (see figure 6, the deformation is calculated at 34, the pattern of the object is 
discriminated at 40 using reference image data as template). One skilled in the art 
would include calculating a deformation value because to include another information 
about the object to improve recognition of the object. 

Ozeki and Okajima do not discuss calibrating the video prior imaging the object. 
However, it is well known to calibrate any sensor, camera or video prior to imaging an 
object or scene. One skilled in the art would include calibrating sensor, camera or video 
because to avoid any pixel or alignment errors that will occur in the capture image, 
reducing the chances of recognition errors or failures. 

With regards to claim 3, Ozeki discloses reflected light is received at a position of the slit 
light receiving unit which is the same as a position of the camera at which said two- 
dimensional image is captured (see figure 1, the slit line detecting unit receives it 
information from the TV camera, which implies the position of the slit light in the image 
is the same as the position of camera). 

With regards to claim 4, Ozeki discloses camera serves as said light receiving unit (see 
citation of claim 3). 

4. Claims 5 - 7 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ozeki ('542) in view of Okajima ('940) and official notice as applied to claim 4 further in 
view of Tsujino (US pat no 5,307,41 9), 
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With regards to claim 5, Ozeki and Okajima disclose all the limitations of claim 4; Ozeki 
discloses combining two dimensional information with three dimensional information 
(see figure 1, lookup table) and Okajima discloses calculating deformation parameter 
(see figure 6, 34), but both references do not disclose mounting camera on a robot and 
captures two and three dimensional information. Tsujino discloses a camera being 
mounted on a robot and captures two dimensional images (column 1, lines 16 to 30, the 
camera is mounted on the autonomous moving body). One skilled in the art would 
include mounting a camera on a robot because operator is able to program positional 
information, knowing where to go next, into the robot, so the operator does not have to 
manually operate it. 

With regards to claim 6, Tsujino discloses obtaining positional information from said 
robot (see column 4, lines 1 to 4). 

With regards to claim 7, Ozeki discloses obtaining three-dimensional means (see figure 
1, and new three dimensional data are obtained because element 14 of figure is a TV 
camera which captures images continuously). 

5. Claims 8-11 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ozeki ('542) in view of Okajima ('940) and official notice as applied to claim 2 further in 
view of Gilliland ('268) and Palm (US pat no 5,699,444). 
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With regards to claim 8, Ozeki and Okajima disclose all the limitations discussed in 
claim 2, but do not disclose means for determining, in the three-dimensional space, a 
straight line which passes through the measuring point on said object and a specific 
point on said camera. Gilliland discloses means for determining, in the three- 
dimensional space, a straight line which passes through the measuring point on said 
object and a specific point on said camera (see figure 4B, R1-1 and R1-2 is read as the 
straight line). One skilled in the art would include determining a straight line which 
passes through a measuring point on the object and a specific point on said camera 
from the image of the object because to use triangulation to find the location of the 
cameras in three dimensional space and determine the shape of the object, to improve 
recognition of the object. 

Ozeki, Okajima and Gilliland do not disclose means for determining, based on 
information on said straight line and the surface on which the measuring point on said 
object exists, an intersection between said surfaces and said straight line. Palm 
discloses means for determining, based on information on said straight line and the 
surface on which the measuring point on said object exists, an intersection between 
said surfaces and said straight line (see figure 10, point A is the camera coordinate, 
point D is read as the object point and point C is read as the intersection point). One 
skilled in the art would include determining an intersection between the object and 
camera plane because to determine the position of the camera, so to use triangulation 
technique to determine three-dimensional shape/coordinate of the object. 
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With regards to claim 9, see the rationale and rejection for claims 2 and 8. 
With regards to claim 10, see the rationale and rejection for claim 3. 
With regards to claim 1 1 , see the rationale and rejection for claim 4. 

6. Claims 12-14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ozeki ('542) in view of Okajima ('940), Gilliland ('268) and Palm ('444) and official notice 
as applied to claim 10 further in view of Tsujino ('419). 

With regards to claim 12, see the rationale and rejection for claim 5. 

With regards to claim 13, see the rationale and rejection for claim 6. 

With regards to claim 14, see the rationale and rejection for claim 7. 

7. Claims 15 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ozeki ('542) in view of Okajima ('940) and official notice as applied to claim 2 
further in view of Kinoshita (US pat no 6,137,902). 
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With regards to claim 15, Ozeki and Okajima disclose all of the claim elements / 
features as discussed above in rejection for claim 2 and incorporated herein by 
reference, but fails to disclose using Affine transform. Kinoshita discloses a visual 
sensor according to claim 2, wherein said transformation is an affine transformation 
(see fig 5). One skill in the art would use Affine transform because affine transforms 
provides linear relation, improving three-dimensional information estimation (see 
Kinoshita col. 1 lines 35 - 40) such as the orientation and position of the camera. 

With regards to claim 16, see the rationale and rejection for claim 15. In addition, affine 
transform obtain perspective model of the camera. 

8. Claims 17 and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ozeki ('542) in view of Okajima ('940), official notice, Gilliland ('268) and Palm 
('444) as applied to claim 9 further in view of Kinoshita ('902). 

With regards to claims 17 and 18, see the rationale and rejection for claims 15 and 16. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Alex Liew whose telephone number is (571)272-8623. 
The examiner can normally be reached on 9:30AM - 7:00PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella can be reached on (571) 272-7778. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Alex Liew 

AU2624 

12/20/07 
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